Sensor shift for remote sensing

ABSTRACT

Techniques for improving the quality of images captured by a remote sensing overhead platform such as a satellite. Sensor shifting is employed in an open-loop fashion to compensate for relative motion of the remote sensing overhead platform to the Earth. Control signals are generated for the sensor shift mechanism by an orbital motion compensation calculation that uses the predicted ephemeris (including orbit dynamics) and image geometry (overhead platform to target). Optionally, the calculation may use attitude and rate errors that are determined from on-board sensors.

BACKGROUND

The use of satellite-based and aerial-based imagery of the Earth ispopular among government and commercial entities. Satellite images maybe collected with multiple different sensors (for example inDigitalGlobe's WV-3 satellite) that at any given instant in time viewdifferent points on the ground. For example, a satellite may containmany separate sensors that are each line scanners.

Each sensor may have one or more bands (e.g., 3-15 bands). Further, oneor more of the sensors may be populated with multispectral VNIR sensors,having a ground resolution of 1.24 meters. VNIR has a fairly standardmeaning in the industry of the portion of the electromagnetic spectrumfrom roughly 400 to 1100 nanometers in wavelength. And multispectralrefers to the use of multiple narrower wavelength ranges throughout therange. For example, it might refer to eight specific wavelength bandswithin the 400-1100 nanometer range (e.g., coastal (approximately400-452 nm), blue (approximately 448-510 nm), green (approximately518-586 nm), yellow (approximately 590-630 nm), red (approximately632-692 nm), red edge (approximately 706-746 nm), near infrared 1 (NIR1)(approximately 772-890 nm), and near infrared 2 (NIR2) (approximately866-954 nm)). Also, one or more of the bands in one or more of the banksmay be populated with panchromatic sensors, having a ground resolutionof 0.31 meters. Panchromatic has a fairly standard meaning in theindustry of a relatively broad spectral band that may include all ormost of the visible spectrum (450 to 700 nanometers) and possibly otherregions adjacent to the visible spectrum (e.g., 450 to 800 nanometers).Also, one or more of the bands in may be populated with SWIR sensors,having a ground resolution of 3.7 meters. SWIR has a fairly standardmeaning in the industry of the portion of the electromagnetic spectrumfrom roughly 1100 to 3000 nanometers in wavelength.

Further, the WV-3 satellite uses a line scanner that is thousands ofpixels wide and has only a few such rows for each of the panchromaticband, several multispectral (MS) bands, CAVIS bands, and so forth.

With an integrated sensor containing each of these sensors, theintegrated sensor field-of-view is typically swept across the Earth'ssurface in “push broom” fashion. Additionally, the attitude (angularposition/orientation) of the satellite may be adjusted to view differentareas on the Earth's surface. Necessarily, many if not all of thedifferent viewing angles will be from a non-nadir position.

As described above, the panchromatic and MS bands each have their ownsection of detectors that simultaneously collect the image. All bandseventually overlay the target area, creating a complete image. Theorbital motion of the satellite is factored into the scan profile, whichresults in an image that covers the target area. Exposure is controlledby the scan rate/line rate as well as the time-delayed-integration,which result in minimal pixel smearing during the scan. Scanning arraysare able to collect a large amount of area in a short time.

Area array image sensors are typically less expensive than line scannerimage sensors, and they capture an image of a much larger ground areathan image sensors. However, because of the amount of time necessary totransfer all of the image data off of an area array image sensor, andbecause of the length of time that a given pixel needs to collectphotons from the Earth's surface, it has previously not been practicalto use area array image sensors for remote sensing from satellites.

These area (or framing) arrays (as what is found in consumer cameras)capture a whole scene in a single snap. They are usually small in imagesize compared to a line scanner, which collects a longer image. Theadvantage of area arrays is that: (1) they are cheaper to buy and alignin the camera during construction; (2) they are smaller, but useful forsmall satellites; and (3) all the pixels are aligned with one another,so it is easier to geo-locate the other pixels once one of the pixels isgeo-located.

However, proper exposure time is needed. Because of the orbital motionof the satellite, the boresight of the camera has to be steadilymaintained on the target during the image for the long-enough exposure,otherwise the image will be smeared.

It is against this background that the improvements disclosed hereinhave been developed.

SUMMARY

Disclosed herein is a remote sensing overhead platform for imaging anarea below the platform. The overhead platform includes a remote sensingoverhead platform body; an image sensor positioned on the remote sensingoverhead platform body so that the image sensor can be moved relative tothe remote sensing overhead platform body in response to controlsignals; and a controller that provides the control signals to the imagesensor for movement relative to the remote sensing overhead platformbody, wherein the control signals are based on movement of the remotesensing overhead platform body relative to the area below to be imaged.

The remote sensing overhead platform may be an orbital satellite. Theimage sensor may be movable in a first plane relative to the remotesensing overhead platform body. The image sensor may be an area arrayimage sensor having a number of rows of pixels that is at leastone-tenth of the number of pixels in each row of pixels.

The control signals may be entirely free of being based on imagecorrelation. The control signals may not be based on image correlation.The control signals may be entirely free of being based on any capturedimage. The control signals may not be based on any captured image.

The image sensor captures an image and wherein the control signals maybe based on predicted orbital motion and the location of the imagerelative to the remote sensing overhead platform. The image sensor maybe an area array image sensor having a number of rows of pixels that isat least one-tenth of the number of pixels in each row of pixels andwherein the image sensor may be moved so as to compensate for motion ofthe remote sensing overhead platform for at least 15 ms.

Also disclosed is an image sensor system carried by a remote sensingoverhead platform for imaging an area below the platform. The imagesensor system includes an image sensor positioned on the remote sensingoverhead platform body so that the image sensor can be moved relative tothe remote sensing overhead platform body in response to controlsignals; and a controller that provides the control signals to the imagesensor for movement relative to the remote sensing overhead platformbody, wherein the control signals are based on movement of the remotesensing overhead platform body relative to the area below to be imaged.

The image sensor may be movable in a first plane relative to the remotesensing overhead platform body. The image sensor may be an area arrayimage sensor having a number of rows of pixels that is at leastone-tenth of the number of pixels in each row of pixels.

The control signals may be entirely free of being based on imagecorrelation. The control signals may not be based on image correlation.The control signals may be entirely free of being based on any capturedimage. The control signals may not be based on any captured image.

The image sensor captures an image and wherein the control signals maybe based on predicted orbital motion and the location of the imagerelative to the remote sensing overhead platform. The image sensor maybe an area array image sensor having a number of rows of pixels that isat least one-tenth of the number of pixels in each row of pixels andwherein the image sensor may be moved so as to compensate for motion ofthe remote sensing overhead platform for at least 15 ms.

Also disclosed is a remote sensing overhead platform for imaging an areabelow the platform. The overhead platform includes a remote sensingoverhead platform body; an image sensor positioned on the remote sensingoverhead platform body so that the image sensor can be moved relative tothe remote sensing overhead platform body in response to controlsignals, wherein the image sensor is movable in a first plane relativeto the remote sensing overhead platform body, wherein the image sensoris an area array image sensor having a number of rows of pixels that isat least one-tenth of the number of pixels in each row of pixels; and acontroller that provides the control signals to the image sensor formovement relative to the remote sensing overhead platform body, whereinthe control signals are based on movement of the remote sensing overheadplatform body relative to the area below to be imaged, wherein thecontrol signals are entirely free of being based on image correlation.The image sensor captures an image and wherein the control signals arebased on predicted orbital motion and the location of the image relativeto the remote sensing overhead platform.

The image sensor may be moved so as to compensate for motion of theremote sensing overhead platform for at least 15 ms.

BRIEF DESCRIPTION OF THE DRAWINGS

The disclosure herein is described with reference to the followingdrawings, wherein like reference numbers denote substantially similarelements:

FIG. 1 is an illustration of a satellite capturing an image of a portionof the Earth, the image being captured from a non-nadir position of thesatellite.

FIG. 2 is an illustration showing a satellite capturing a series ofswaths of ground on the Earth with a line scanner image sensor operatingin push-broom fashion.

FIGS. 3A and 3B show a comparison of an area of ground captured by aline scanner image sensor as compared to an area of ground captured byan area array image sensor.

FIG. 4 is an illustration showing a satellite capturing a series ofareas of ground on the Earth with a area array image sensor.

FIG. 5 is an illustration of sensor shift in a remote sensing satellite,where an area array image sensor is continuously shifted to compensatefor satellite motion.

FIG. 6 is a block diagram showing relevant portions of a remote sensingsatellite, which employs sensor shift of an area array image sensor.

FIG. 7 shows a control structure demonstrating the techniques taughtherein.

DETAILED DESCRIPTION

While the embodiments disclosed herein are susceptible to variousmodifications and alternative forms, specific embodiments thereof havebeen shown by way of example in the drawings and are herein described indetail. It should be understood, however, that it is not intended tolimit the invention to the particular form disclosed, but rather, theinvention is to cover all modifications, equivalents, and alternativesof embodiments of the invention as defined by the claims. The disclosureis described with reference to the drawings, wherein like referencenumbers denote substantially similar elements.

This disclosure relates to several aspects for improving techniques forremote sensing from satellites and other above-ground imaging locationsusing sensor shift technology. Sensor shift is the movement of theelectronic image sensor in consumer digital cameras (e.g., in DigitalSLRs) to compensate for motion of the camera, such as might be caused bythe camera user not holding the camera sufficiently still (such as mayoccur at long focal lengths). In general, sensor shift may be used as analternative to, or as an adjunct to other types of optical imagestabilization techniques, such as varying the optical path to the sensorsuch as by moving one or more lens elements in the optical path. Butsensor shift can also be used as the sole means of image stabilization.

For example, as taught herein for satellite applications and otherremote sensing from an overhead platform applications, sensor shifttechniques could be used in order to compensate for orbital motion ofthe sensing platform (e.g., the satellite) or to compensate for attitudeerror (movement of the sensing platform due to operations such asslewing the sensing platform to be directed at a particular target orarea on the ground).

This technique would use sensor shift to compensate for the predictedorbital motion. By not having to maneuver/re-point the satelliteboresight for every adjacent image, one can increase the amount ofimagery that can be collected over a short period of time. The shiftingsensor can also be used to adjust fine-pointing of the image, ratherthan relying on the satellite maneuvering to fine-tune the imagepointing.

FIG. 1 shows an illustration of a satellite 100 capturing an image of aportion 102 of the Earth. As can be appreciated, the image is beingcaptured from a non-nadir position. The orientation of the satellite 100relative to the Earth can be changed by the satellite, or it may remainfixed for some portion of time. Further, the images may be capturedsequentially as the area on the Earth imaged by the satellite changes inpush-broom fashion. This is shown in FIG. 2, where the satellitecaptures a series of swaths of ground on the Earth with a line scannerimage sensor operating in push-broom fashion. Although shown with a linescanner image sensor, a similar approach could be used with an areaarray image sensor.

FIGS. 3A and 3B show a comparison of an area of ground captured by aline scanner image sensor as compared to an area of ground captured byan area array image sensor. For example, a line scanner image sensor mayhave 3 to 5 rows of pixels with thousands of pixels in each row. Thus,the “area” imaged by the line scanner image sensor may be in the rangeof 15 m×45 km. Also, by way of example only, the area array image sensormay include (in the range of) 8,000 rows of pixels with (in the rangeof) 15,000 pixels in each row. Thus, the “area” imaged by area arrayimage sensor may be in the range of 24 km×45 km. It should be understoodthat these specific numbers of pixels and specific areas are onlyexamples, and that the techniques taught herein are applicable to othersizes of pixel arrays as well. One parameter that could be used todistinguish an area array from a line scanner might be that with an areaarray has an aspect ratio no greater than ten to one (10:1), meaningthat there are no more than ten times the number of pixels in each rowthan there are rows of pixels. Stated alternatively, the number of rowsof pixels is at least one-tenth ( 1/10) of the number of pixels in eachrow.

As a further specific example that is not intended to be limiting to theinvention described herein, it may be desirable to allow pixels of animage sensor to collect light from a given target (such as a given areaon the surface of the Earth) for 30 milliseconds (ms) in order tocapture an image of a desired quality at a particular light level.Further, at a typical orbit speed and altitude for one of DigitalGlobe'ssatellites, a given spot on the Earth's surface below the satellite mayappear to move relative to the satellite at a rate of approximately 7km/sec. Following along with this example, an image sensor would need tobe moved approximately 210 meters to cause the image sensor and theEarth to appear to have no relative movement for the desired 30 ms. Inthe case of an image sensor with pixels that effectively capture an areaon the ground that is 3 meters long, this would require the image sensorto be shifted approximately 70 pixels. This amount of “sensor shift”would amount to less than 1% of either dimension of the 8,000×15,000pixel sensor described above. It should be understood that 30 ms is butone example of a desirable exposure time. 15 ms is another example of adesirable exposure time. The actual exposure time needed is a functionof the sensor used and the environmental conditions.

FIG. 4 is an illustration showing a satellite capturing a series ofareas of ground on the Earth with an area array image sensor. As can beseen, the area array image sensor could capture an image of Area A onthe Earth's surface, followed by an image of adjacent Area B on theEarth's surface, followed by an image of adjacent Area C on the Earth'ssurface. As could be appreciated, Areas A, B, and C could be slightlyoverlapping, so that Area B includes a small portion that was alsoimaged by Area A, and Area C includes a small portion that was alsoimaged by Area B.

FIG. 5 is an illustration of sensor shift in a remote sensing satellite500, where an area array image sensor 502 is continuously shifted tocompensate for satellite motion. As can be seen, at Time t₁ the imagesensor 502 is in Position A relative to the satellite 500. Later, atTime t₂ the image sensor 502 is in Position B relative to the satellite500. Still later, at Time t₃ the image sensor 502 is in Position Crelative to the satellite 500. Finally, at Time t₄ the image sensor 502is again in Position A relative to the satellite 500, and the sequenceof moving through Positions A, B, and C can be repeated. The intent isthat at each of Time t₁, Time t₂, and Time t₃, the image sensor 502 iscapturing image data from the same point on the Earth's surface, whileat Time t₄ the image sensor 502 is capturing image data from a roughlyadjacent point on the Earth's surface. It should be understood that thisbut a very simplified example. For example, the image sensor wouldlikely be moved through many more positions than Positions A, B, and C.As may be appropriate, the image sensor may be moved continuouslythrough a range of positions or moved discretely through multiplepositions. Also, it should be understood that the movement of the imagesensor may be more complex that simple movement along a single axis(e.g., along an x-axis), such as one might infer from this example.Instead, the sensor may be movable within an x-y plane, both in straightand curved lines within the plane and rotational movement within theplane, and combinations thereof. These types of movement may bebeneficial for off-nadir imaging as well as to adjust for attitudeerrors during imaging.

FIG. 6 is a block diagram showing relevant portions of a remote sensingsatellite 600, which employs sensor shift of an area array image sensor602. As can be seen, the satellite 600 also includes a controller 604(which may include a single controller or multiple differentcontrollers) and various components for the satellite to use indetermining the position and orientation in space of the satellite andthe position and orientation of the satellite relative the Earth. Thesecomponents may include one or more GPS sensors 606, one or moregyroscopes 608, and one or more star trackers 610. As can beappreciated, this block diagram omits numerous conventional satellitecomponents, for ease of illustration and understanding.

FIG. 7 shows a control structure 700 demonstrating the techniques taughtherein. A sensor shift mechanism 702 is shown to be movable in any oneor a combination of an x-direction, a y-direction, and a rotation in thex-y plane. The sensor shift mechanism 702 receives control or movementcommands (in the form of x, y, and z shift profiles) from an orbitmotion calculation 704. The orbit motion calculation 704 receives thefollowing inputs: image geometry (the satellite to the target portion ofthe Earth's surface) 706, predicted ephemeris/orbit dynamics 708, and(optionally) attitude and rate errors 710 calculated from on-boardsensors (e.g., gyroscopes and star trackers).

While there arguably may have been some use of sensor shifting insatellites, the shifting was controlled with much more expensive andcomplex techniques which use optical or image correlation or at leastuse some form of analysis of captured images. The image correlation isused to calculate the required sensor shift—two images are captured, andthe “shift” between the two images is calculated based on correlatingthe two images and determining the movement between the two images. Thenthe sensor is shifted to compensate for that calculated motion. It isimportant to note that our techniques taught herein do not require oruse optical correlation or any other type of image analysis.

Our techniques include:

1) using the predicted orbit motion (which is very accurate) and usingthe image geometry (location of image relative to satellite) tocalculate the sensor shift profile. This profile is then used to commandthe sensor shift mechanism to move in x, y, z, as needed, over theduration of the image capture (e.g., 15-30 milliseconds) to maintain asteady image attitude on the target. Real-time, closed-loop, motioncompensation (via sensors or image correlation) is not needed. Rather,open loop calculation can be made using the highly accurate ephemerisalready readily available to the satellite and the satellite-to-targetgeometry (which geometry is readily available). Orbit motioncompensation is extremely predictable and accurate using ephemeris andimage geometry.

2) using the on-board attitude and rate sensors (gyros, star trackers,and so forth) to calculate the error in attitude and rate from thedesired attitude and rate. Rather than using maneuvering of thesatellite to get these errors to within acceptable tolerances, we usethe satellite maneuvering to get close, and then use sensor shift tofine-tune the reduction of these errors to within tolerance. The benefitis faster time to adjust the attitude from image to image.

While the embodiments have been illustrated and described in detail inthe drawings and foregoing description, such illustration anddescription are to be considered as examples and not restrictive incharacter. For example, certain embodiments described hereinabove may becombinable with other described embodiments and/or arranged in otherways (e.g., process elements may be performed in other sequences).Accordingly, it should be understood that only example embodiments andvariants thereof have been shown and described.

I claim:
 1. A remote sensing overhead platform for imaging an area belowthe platform, comprising: a remote sensing overhead platform body; animage sensor positioned on the remote sensing overhead platform body sothat the image sensor can be moved relative to the remote sensingoverhead platform body in response to control signals; and a controllerthat provides the control signals to the image sensor for movementrelative to the remote sensing overhead platform body, wherein thecontrol signals are based on movement of the remote sensing overheadplatform body relative to the area below to be imaged, wherein thecontrol signals are calculated from a predicted orbit motion, andwherein the calculation of the control signals fails to utilize each ofoptical correlation and image analysis.
 2. A remote sensing overheadplatform as defined in claim 1, wherein the remote sensing overheadplatform is an orbital satellite.
 3. A remote sensing overhead platformas defined in claim 1, wherein the image sensor is movable in a firstplane relative to the remote sensing overhead platform body.
 4. A remotesensing overhead platform as defined in claim 1, wherein the imagesensor is an area array image sensor having a number of rows of pixelsthat is at least one-tenth of the number of pixels in each row ofpixels.
 5. A remote sensing overhead platform as defined in claim 1,wherein the image sensor captures an image and wherein the controlsignals are based on the predicted orbital motion and the location ofthe image relative to the remote sensing overhead platform.
 6. A remotesensing overhead platform as defined in claim 1, wherein the imagesensor is an area array image sensor having a number of rows of pixelsthat is at least one-tenth of the number of pixels in each row of pixelsand wherein the image sensor is moved so as to compensate for motion ofthe remote sensing overhead platform for at least 15 ms.
 7. An imagesensor system carried by a remote sensing overhead platform for imagingan area below the platform, the image sensor system comprising: an imagesensor positioned on the remote sensing overhead platform body so thatthe image sensor can be moved relative to the remote sensing overheadplatform body in response to control signals; and a controller thatprovides the control signals to the image sensor for movement relativeto the remote sensing overhead platform body, wherein the controlsignals are based on movement of the remote sensing overhead platformbody relative to the area below to be imaged, wherein the controlsignals are calculated from a predicted orbit motion of the platform,and wherein the calculation of the control signals fails to utilize eachof optical correlation and image analysis.
 8. An image sensor system asdefined in claim 7, wherein the image sensor is movable in a first planerelative to the remote sensing overhead platform body.
 9. An imagesensor system as defined in claim 7, wherein the image sensor is an areaarray image sensor having a number of rows of pixels that is at leastone-tenth of the number of pixels in each row of pixels.
 10. An imagesensor system as defined in claim 7, wherein the image sensor capturesan image and wherein the control signals are based on the predictedorbital motion and the location of the image relative to the remotesensing overhead platform.
 11. An image sensor system as defined inclaim 7, wherein the image sensor is an area array image sensor having anumber of rows of pixels that is at least one-tenth of the number ofpixels in each row of pixels and wherein the image sensor is moved so asto compensate for motion of the remote sensing overhead platform for atleast 15 ms.
 12. A remote sensing overhead platform for imaging an areabelow the platform, comprising: a remote sensing overhead platform body;an image sensor positioned on the remote sensing overhead platform bodyso that the image sensor can be moved relative to the remote sensingoverhead platform body in response to control signals, wherein the imagesensor is movable in a first plane relative to the remote sensingoverhead platform body, wherein the image sensor is an area array imagesensor having a number of rows of pixels that is at least one-tenth ofthe number of pixels in each row of pixels; and a controller thatprovides the control signals to the image sensor for movement relativeto the remote sensing overhead platform body, wherein the controlsignals are based on movement of the remote sensing overhead platformbody relative to the area below to be imaged, wherein the controlsignals are entirely free of being based on each of image correlationand image analysis; wherein the image sensor captures an image andwherein the control signals are based on predicted orbital motion andthe location of the image relative to the remote sensing overheadplatform.
 13. A remote sensing overhead platform as defined in claim 12,wherein the image sensor is moved so as to compensate for motion of theremote sensing overhead platform for at least 15 ms.